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RATE-BASED MULTI-LEVEL ACTIVE QUEUE MANAGEMENT 
WITH DROP PRECEDENCE DIFFERENTIATION 

RELATED U.S. APPLICATION DATA 

[0001] This patent application is a continuation-in-part application of U.S. Patent 
Application No. 10/426,763 (Attorney Docket No. 120-225) [15646RO] filed April 30, 
2003; U.S. Patent Application No. 10/426,286 (Attorney Docket No. 120-226) [15645RO] 
filed April 30, 2003; and U.S. Patent Application No. 10/426,289 (Attorney Docket 
No. 120-227) [15738RO] filed April 30, 2003, the contents of which are hereby 
incorporated by reference. 

FIELD OF THE INVENTION 

[0002] The present invention relates generally to network queue management 
and, more particularly, to a technique for rate-based multi-level active queue 
management with drop precedence differentiation. 

BACKGROUND OF THE INVENTION 

[0003] The networking community is currently seeking mechanisms which will 
enable networks to provide service differentiation based on application and user needs 
while simultaneously managing congestion. 

[0004] Congestion control in the Internet has primarily been the responsibility of 
the end-to-end congestion control mechanisms of TCP (Transmission Control Protocol). 
However, with the rapid growth of the Internet and the stringent requirements for quality 
of service (QoS) support, it has become clear that the Internet could not exclusively rely 
on the end hosts to perform the end-to-end congestion control. Mechanisms are needed 
in the intermediate network elements to complement the end hosts congestion control 
mechanisms. Recognizing this, the Internet Engineering Task Force (IETF) has 
advocated the deployment of active queue management (AQM) mechanisms at the 
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intermediate network elements (routers, switches, etc.) as a means of congestion 
control. 

[0005] To perform AQM, the network elements are equipped with the means to 
detect incipient congestion and to signal the traffic sources before congestion actually 
occurs. AQM mechanisms allow the network elements to send explicit/implicit feedback 
of congestion to the end hosts by marking/dropping packets. The end hosts in turn react 
to the packet marking/dropping by reducing their data transmission rates. The main 
goals of AQM are to reduce the average queue lengths in the network elements and 
thereby decrease the end-to-end delay experienced by end user traffic, and maximize 
the utilization of network resources by reducing the packet loss that occurs when 
queues overflow. 

[0006] The current Internet and most private corporate networks offer best-effort 
service to traffic. In this model, all user packets compete equally for network resources. 
The best-effort service model has been sufficient until recently when usage and 
popularity of IP networks (e.g., the Internet) soared. This rise in usage and popularity 
has been paralleled by a rise in user expectation regarding the type of applications and 
quality of service offered by these networks. The networking community has recognized 
the importance of simplicity in packet forwarding mechanisms, but also the inadequacy 
of the best-effort service model in supporting the wide range of applications and user 
requirements. Different applications and users are increasingly demanding different 
quality of service (QoS) and network usage rates. 

[0007] To address these new demands, the IETF has been looking at a number 
of architectural enhancements to enable networks to provide service differentiation 
based on application and user needs. One of these efforts has resulted in the 
differentiated services (DiffServ) architecture (as described in K. Nichols, S. Blake, F. 
Baker, and D. Black, " Definition of the Differentiated Services Field (PS Field) in the 
IPv4 and IPv6 Headers ." IETF RFC 2474, Dec. 1998; and in S. Blake, D. Black, M. 
Carlson, E. Davies, Z. Wang, W. Weiss, " An Architecture for Differentiated Service ," 
IETF RFC 2475, Dec. 1 998). 



-3- 



[0008] DiffServ aims to provide a scalable framework for supporting various levels 
of service to different users and the pricing of such services. It is an approach to service 
level specifications in IP networks based on traffic conditioning at the network edge and 
differentiated forwarding of aggregated traffic flows at the network core. The underlying 
goal of t he D iffServ a rchitecture i s to a ddress t he s calability i ssue regarding p er-flow 
service differentiation in the core of the network. In DiffServ, the core network elements 
do not necessarily have to implement complex resource reservation, scheduling, 
processing, and classification mechanisms in addition to maintaining state information 
for each individual traffic flow. The architecture allows network designers to push most 
of the state and forwarding complexity to the edge of the network. 
[0009] In DiffServ, a distinction is made between boundary and interior nodes. 
Boundary nodes classify incoming traffic into a small number of aggregate flows 
according to their diverse QoS requirements such as bandwidth, delay, and packet drop 
precedence. Once the traffic enters the network, interior nodes deal only with the 
aggregated traffic. A DiffServ domain refers to a contiguous set of nodes operating with 
a common set of service provisioning policies and per-hop-behavior (PHB) definitions. 
Per-domain sen/ices are realized by traffic conditioning at the edge and simple 
differentiated forwarding at the core of the network. Packets are marked with the 
appropriate DiffServ code point (DSCP) at the edge of the network and within the core of 
the network, the network nodes (routers, switches, etc.) simply forward packets based 
on the PHB associated with the DSCP. An end-to-end differentiated service is obtained 
by concatenation of per-DiffServ domain services. 

[0010] Two forwarding mechanisms recently standardized by the IETF are 
Expedited Forwarding (EF) PHB and Assured Forwarding (AF) PHB. The purpose of the 
EF PHB is to support low loss, low latency, low jitter, assured bandwidth, end-to-end 
premium service. These services can be used to support delay sensitive traffic requiring 
high service guarantees. In order to provide the necessary assurance, sufficient network 
resources need to be reserved. The EF PHB provides a point-to-point connection or a 
"virtual leased line" type of service. The AF PHB standard specifies four classes and 
three levels of drop precedence per class. Packets entering the network are classified 
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as belonging to one of these four traffic classes. The three drop precedence levels (from 
high to low priority) are also referred in terms of color as green (DPO), yellow (DP1), and 
red (DP2). In case of congestion, an AF-compliant DiffServ node drops low precedence 
(red) packets in preference to higher precedence (green, yellow) packets. The AF PHB 
group is intended to provide a flexible, low-cost support for a wide range of traffic needs. 
[0011] The AF drop precedence levels (or colors) are implemented using an 
active queue management (AQM) algorithm in the network nodes. AQM has been 
proposed in B. Braden, et al, " Recommendation on Queue Management and 
Congestion Avoidance in the Internet ." IETF RFC 2309, Apr. 1998, as a means to 
provide some congestion control as well as some notion of QoS to users. 
[0012] One important class of AQM is based on randomized packet dropping or 
marking. With this form of AQM, a network node drops each arriving packet with a 
certain probability, where the exact probability is a function of the average queue size or 
any suitable indicator of network congestion such as rate mismatch at a node. The idea 
behind AQM is to detect incipient congestion early and convey congestion notification to 
the end-hosts, allowing them to back off before queue overflow and sustained packet 
loss occur. The AQM algorithm has to detect and respond to long-term congestion by 
discarding or marking packets. Very small buffers tend to complicate the control problem 
in this case as the short queues more easily overflow and do not provide adequate 
information for process control. 

[0013] In view of the foregoing, it would be desirable to provide a technique for 
network queue management which overcomes the above-described inadequacies and 
shortcomings by providing a mechanism which does not react to short-term burst traffic 
and allows each precedence level to be addressed differently. During long-term 
congestion the highest priority traffic should receive the lowest discard rate whereas the 
lowest priority traffic should receive the highest discard rate. 
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SUMMARY OF THE INVENTION 

[0014] An object of the present invention is to provide a mechanism for improved 
rate-based multi-level active queue management with drop precedence differentiation. 

[0015] According to an aspect of the invention, there is provided a method for 
controlling a data flow in a data network, starting with setting a stable integral controller 
gain k h Next specifying a plurality of precedence grades, each of the precedence 
grades has an associated priority. Then, for each precedence grade measuring a 
cumulative data arrival rate R(n) at time n, where R(n) is the sum of the data arrival rates 
for a particular precedence grade under consideration plus the data arrival rates of all 
precedence grades with a higher priority than the particular precedence grade under 
consideration. Next, there is calculated for each precedence grade a normalized error 
signal e(n), according to the relation e(n) = (T(n) - R(n))/x, where T(n) is a capacity at 
time n, and x is a nominal packet size. Following this, there is computed for each 
precedence grade a mark/drop probability p(n) according to the relation 

p(n) = min { max [p(n-1 ) + k r At • e(n), 0 ] , p max } 
where At is the time interval between a (n-1) th and the n th computation, and 0 < p max ^ 1 ; 
and, subsequently, for each precedence grade executing a packet mark/drop routine 
based upon the calculated mark/drop probability p(n). 

[0016] Conveniently, the data arrival rate may be filtered by use of an 
exponentially weighted moving average scheme according to the relation: 

R c '(n)= (1-p).R c '(n-1) + p.R c (n) 

where (5 is a filter gain parameter such that 0 < p < 1, 

Rc(n-1) is the filtered data arrival rate at time n-1, 

R c '(n) is the desired filtered data arrival rate at time n, and 

R c (n) is the cumulative data arrival rate at time n. 
[0017] According to another aspect of the invention, preceding the packet/mark 
drop routine may be a bypassing routine involving the steps of testing the cumulative 
data arrival rate R c (n) for each precedence grade against a rate threshold T L specific to 
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that precedence grade, and if the data arrival rate R(n) is below or equal to the rate 
threshold Tl then bypassing the step of executing a packet mark/drop routine. 
[0018] Conveniently, the packet mark/drop routine may be realized according to a 
random number generator mark/drop scheme. 

[001 9] The s table i ntegral c ontroller g ain m ay b e c hosen f rom a p re-calculated 
regime of stable gains. A method for pre-calculating the regime may be as follows: 
starting with (1 ) obtaining for said network a value for said network a set of parameters 
k, d 0 , and x, where £ is a steady-state gain of said network, d 0 is a time delay of said 
network, and x is a time constant of said network. Next step (2) determining the root z/ 

in the interval 0,— of cot(zj) = — z, t anc j then computing the range of stable gains 

V 2) "o 

for ki for this particular network as:\ 



Ml "T 1 r 2 

[0020] In accordance with another other aspect of the present invention, there is 
provided an apparatus for rate-based multi-level active queue management with drop 
precedence differentiation, the apparatus being configured according to the methods 
described above. 

[0021] In accordance with another other aspect of the present invention, there is 
provided an apparatus for controlling a data flow in a data network. The apparatus has 
an integral controller having an integral controller gain £. setting for which the said 

network is stable, and a cumulative data rate calculator for calculating a cumulative data 
arrival rate R(n) associated with each of said plurality of precedence grades, wherein 
R(n) is the sum of the data arrival rates for a particular precedence grade under 
consideration plus the data arrival rates of all precedence grades with a higher priority 
than said particular precedence grade under consideration. The apparatus further 
contains an error signal calculator for calculating a normalized error signal e(n) for each 
of said plurality of precedence grades according to the relation 
e(n) = (T(n) - R(n))/x, 
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where T(n) is an assigned precedence grade capacity at time n, and x is a nominal 
packet size. Further, the apparatus has a mark/drop probability processor for computing 
a mark/drop probability p(n) for each of said plurality of precedence grades according to 
the relation 

p(n) = min { max [p(n-1 ) + £,-• At • e(n), 0 ] , p max } 
where At is the time interval between a (n-1) th and the n th computation, and 0 < p ma x^ 1 ; 
and a packet mark/drop module for executing a packet mark/drop routine based upon 
the calculated mark/drop probability p(n). 

[0022] In accordance with another other aspect of the present invention, there is 
provided an article of manufacture carrying instructions for a method for rate-based 
multi-level active queue management with drop precedence differentiation in a data 
network and, further, there is provided a signal embodied in a carrier wave representing 
instructions for a method for rate-based multi-level active queue management with drop 
precedence differentiation in a data network according to an integral control scheme. 
[0023] The present invention will now be described in more detail with reference 
to exemplary embodiments thereof as shown in the appended drawings. While the 
present invention is described below with reference to the preferred embodiments, it 
should be understood that the present invention is not limited thereto. Those of ordinary 
skill in the art having access to the teachings herein will recognize additional 
implementations, modifications, and embodiments which are within the scope of the 
present invention as disclosed and claimed herein. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

[0024] The invention will be further understood from the following detailed 
description of embodiments of the invention and accompanying drawings in which: 

[0025] FIG. 1 is a diagram of an Active Queue Management controller in an 
Intermediate Network Element according to an embodiment of the 
invention. 

[0026] FIG. 2 is a diagram of a closed-loop control system according to an 

embodiment of the invention. 
[0027] FIG. 3 is a plot of the two terms in Equation (27) for a general system. 
[0028] FIG. 4 is a block diagram of a routine for determining the stable integral 

gains for a closed-loop TCP/AQM system according to an embodiment of 

the invention. 

[0029] FIG. 5 is a plot of an example of the real and imaginary parts of F*(jco) 
according to an example embodiment of the invention. 

[0030] FIG. 6 is a plot of the step response of an example Integral Controller 
according to FIG. 2. 

[0031] FIG. 7 is a block diagram of a mark/drop probability computation routine 

according to an embodiment of the invention. 
[0032] FIG. 8 is a block diagram of a packet drop/mark routine according to an 

embodiment of the invention. 
[0033] FIG. 9 is a plot of the relative relationship of three rate thresholds 

according to an embodiment of the invention. 
[0034] FIG. 10 is a plot of the relative relationship of three drop probabilities 

according to an embodiment of the invention. 
[0035] FIG. 11 is a block diagram of a three threshold mark/drop probability 

computation according to an embodiment of the invention. 
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[0036] FIG. 12 is a block diagram of a multiple precedence grade packet 
mark/drop routine according to an embodiment of the invention. 

[0037] FIG. 1 3 is a plot of the relative relationship of three rates relative to a 
single rate mark/drop threshold according to an embodiment of the invention. 
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DETAILED DESCRIPTION 

[0038] The description that follows describes a rate-based AQM scheme which 
matches the aggregate rate of the active TCP connections to the specified available 
capacity while maintaining minimal queue size and high-link utilization. Following the 
description of the basic mechanism, an enhanced version is described which details an 
embodiment which allows for precedence grade differentiation. 

[0039] A high-level depiction of an implementation of the active queue 
management integral control scheme according to a possible embodiment is given in 
FIG.1. 

[0040] Users 100 communicate with End Network Elements 110 which pass 
packet data either directly or through Intermediate Network Elements 120 to other End 
Network Elements 110. End-to-end congestion control mechanisms under normal TCP 
rely upon Implicit/Explicit Feedback Signals 105. Examples of Implicit/Explicit Feedback 
Signals 1 05 typically include duplicate TCP acknowledgement packets, timeouts, and 
marked packets. 

[0041] In order to meet the stringent requirements for Quality of Service (QoS) of 
users and the increases in data volume with the rapid growth of the Internet, it is 
advantageous to provide congestion control mechanisms at intermediate points to 
complement the end hosts' congestion control mechanisms. According to one aspect of 
the invention, Intermediate Network Element 120 has been equipped with such a 
mechanism for congestion control. 

[0042] Within Intermediate Network Element 120 are a Integral Controller 121, a 
queue means 126, an arrival rate measurement means 122, a target rate measurement 
means 124, a packet mark/drop module 125, and a mark/drop probability control means 
123 by which the Integral Controller 121 influences the packet mark/drop module 125 
according to a scheme described hereafter. 

[0043] A control system oriented depiction of the system of FIG. 1 may be seen in 
FIG. 2 where the closed-loop system is represented as having two main elements: 
TCP/AQM plant 210 and the control-function means 220. The control system functions 
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by comparing a desired Target Rate T 226 with an Actual Rate "/?" 222 via a summer 
240 which generates a control error "e" 250 by determining the difference between the 
two rates. This difference is used as an input to the control-function means 220 which 
generates a mark/drop probability threshold "p" 224. Changes in the threshold for 
marking/dropping packets results in an eventual change in the Actual Rate U R' 222 as 
the effects propagate through the system. As with all closed-loop systems, the control- 
function means 220 must have a control function response chosen so as to render a 
stable closed-loop system response. If this condition is not met, the resultant 
performance will be deleterious to the network operation, possibly to the point of causing 
performance degradation. 

[0044] In order to be able to determine the stability requirements for the control- 
function means, the open-loop transfer function for TCP/AQM plant 210 must be 
established. This may be done as follows. 

[0045] A simplified rate-based fluid-flow model of a TCP source / assessing a 
single link is described by the following nonlinear differential equation: 

m = -1- - Qr, (Or, {t - d, (t))p(t - d % (/)) ( 1 ) 

Of (0 

where & denotes the time derivative of r, r { is rate of the TCP source (in packets per 

second), d i is the round-trip time (RTT) of the TCP source (in seconds), p is the 

dropping probability of a TCP packet at the link, and 0 is a parameter characteristic of 
the type of TCP source. This simplified model of TCP ignores the slow-start and the 
timeout behavior of TCP. Note that r. and p are bounded, respectively, as follows: 

r { e [0,C] , p € [0,1] , where C is the link capacity. 

[0046] Denoting p 0 as the steady-state (or operating point) of the dropping 
probability of a TCP source and assuming a constant RTT d 0 , the steady-state 
throughput r 0 can be obtained as: 
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O = -^-0r o 2 p o or 



r o = 



" — 



[0047] If the parameter 0 takes on the value of 2/3, the steady-state throughput 
of TCP is obtained as: 



r ° = doiZo' 



[0048] Making the approximation relating data transmission rate and window size 

/ , WXt) 
r(0«— — , 

and assuming d,(t) = d t = constant , it obtains that: 



f W t (0\ 1 dW ( (0 W^ddi 1 dW.it) 



) d, dt df dt d t dt 

With this, it can be seen that making the approximation r i (t)^W i (0/d i (0 shows that 

the rate-based fluid-flow model of a TCP source is essentially equivalent to a window- 
based, nonlinear, fluid-flow TCP model. 

[0049] Assuming a constant RTT d 0 , rewrite the initial equation as: 



An , r w , p) = m = -L - 0r { (0r, d (0p(t -d 0 ) (2) 



where 



r u (0 </<>)• 

[0050] Using Taylor series expansion, the small-signal linearization of 

f(r i9 r. d ,p) = i8(0 
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about an operating point (r 0 ,/? 0 ) is given as 



where 



and 



[0051] 



r o,Po 



dr, 



*w(0 + 



r o.Po 



dp 



Sp{t-d,) 



ro.Po 



8r.(t) = ri (t)-r 0 
Sp(t-d 0 ) = p(t-d 0 )-p 0 



¥ 



¥ 



= -0r o p o 



'b.Po 



^ - fir 2 



r o,Po 



The linearized equation (3) then becomes 



(3) 



(4) 



[0052] Making the approximation r.(t) * r w (r) in (4), it follows that 

S$t) = -20r oPo Sr t rt) - 0r^Sp(t - d 0 ) 



(5) 



[0053] Assume that the link of capacity C is shared by a constant number N of 
TCP sources. The aggregate rate on the link R(t) is given by 



2>,(O = *(0. (6) 
[0054] From (6), the equilibrium point of the system can be expressed as 

jTr 0 =R 0 = pC 9 (7) 
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or 

*-f. (8) 

where pe(p,\] is a target utilization factor. From (1) and (8), the equilibrium point of the 
marking/dropping probability can be expressed as 



or 



1 N 2 

Po ~ d\eri ~ 0(d oP cf ■ W 



[0055] Defining SR(t) = R(t) -R 0 ,\t is known from (6) that 

ffl(0 = j> ( (0 ( 10 > 



and 

= -2er 0 p o j^S ri (t)-Zer'Sp(t-do) (11) 

1=1 1=1 

= -20r oPo SR(t) - MrtSpit - d 0 ) 
[0056] Substituting the equilibrium points (r 0 ,p 0 ) in (1 1 ), it follows that 



dlpC N 



<)'--Sb.m)-^-Sp(l-d,) (12) 



TSR\t) + 6R(t) = -k8p(t-d 0 ) 



d\fC . , 0dlp 3 C 3 
where r = and A; = — 

2W 2W 2 
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[0057] Taking the Laplace transform of (1 3) it obtains that 

tsM(s) + 5R(s) = -ke d «P{s) , (14) 

where 9*(j) is the Laplace transform of SR(t) and e'^'P^s) is the Laplace transform of 

Sp(t-d 0 ). 

[0058] The open-loop transfer function of the TCP/AQM plant may be obtained 
from (14) as 

.91(5) z ke^_ M . 

G ( s ) = TT7 = — — • (15) 
P(s) ts + l 

[0059] In (15), -k represents the steady-state (or static) gain of the TCP/AQM 
plant, d 0 represents the time delay (or dead time), and r represents the time constant 
of the plant. 

[0060] Returning to FIG. 2 and noting again that Target Rate T 226 is the control 
target, Actual Rate "RT 222 is the output of the plant (actual traffic arrival rate), 
mark/drop probability threshold "p" 224 is the control input, TCP/AQM plant 210 given by 
(15) as the plant to be controlled, and C(s) , the control-function means 220 is to be a 
integral controller. 

[0061] The Laplace transform expression for the case of a integral controller may 
be written as: 

C(s) = ^, (16) 
s 

where is integral gain of the controller. 

[0062] The closed-loop transfer function of the TCP/AQM plant is then given as 

C(s)G(s) _ -kk^- 
" abAS) l + C(s)G(s) ~ is 2 + s-kk i e- d °> • (1 f) 



-16- 



[0063] The objective now is to determine the values of the parameter k t for which 
the closed-loop system is stable. 

[0064] Using an extension of the Hermite-Biehler Theorem applicable to 
quasipolynomials a detailed analytical characterization of the stabilizing feedback gains 
of the closed-loop TCP/AQM system may be performed as follows. 

[0065] The characteristic equation of a control system with time delay can be 
expressed in the general form as 

F(s) = d(s) + e-' T %(s) + e- sT2 n 2 (s)+K +e sT *n m {s) 9 (18) 
where d(s) , n.(s) for / = 1,2,K ,w are polynomials with real coefficients. 

[0066] Characteristic equations of the form (18) are also referred to as 
quasipolynomials. It can be shown that the Hermite-Biehler Theorem for Hurwitz 
polynomials does not carry over to arbitrary functions f(s) of the complex variable s. 
However, a suitable extension of the Hermite-Biehler Theorem can be developed to 
study the stability of certain classes of quasipolynomials characterized as follows. 

[0067] If the following assumptions are made in (18) 

A1 : dcg[d(s)] = n and deg[« ( .(s)] < n for i = 1,2,K 9 m ; 

A2: 0<7;<r 2 <K <T m , 

then instead of (18), one can consider the quasipolynomial 

F*(5) = e sT "F(s) = e 5r *rf(s) + e 5(r «- r '^ +n m (s) . (19) 

[0068] Since e sTm does not have any finite roots, the roots of F(s) are identical to 
those of F*(s). The quasipolynomial i 7 *^), however, has a principal term, i.e., the 

coefficient of the term containing the highest powers of s and e s is nonzero. It then 
follows that this quasipolynomial is either of the delay (i.e., retarded) or of the neutral 
type. From this, it follows that the stability of the system with characteristic equation (18) 
is equivalent to the condition that all the roots of F*(s) be in the open left-half plane. 
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Equivalently, it can be stated that F*(s) is Hurwitz or stable. The theorem below gives 
necessary and sufficient conditions for the stability of F*(s) : 

Theorem 1: Let F*(s) be given by (19), and write 

F*(ij&) = F r (a>) + jF^o) 

where F r (co) and F^co) represent the real and imaginary parts of F*(s), 
respectively. Under assumptions A1 and A2, F*(s) is stable if and only if 

1) F r (o>) and F^a) have only simple real roots and these interlace 

2) ^rW-^W^o) > 0, for some co 0 in (-00,00); 

where 7&(<y) and 1&(cd) denote the first derivative with respect to co of F r (co) and 
F % {co) , respectively. 

[0069] Using this theorem, the set of all integral gains that stabilize the first-order 
TCP/AQM plant with time delay described by (15) may be characterized. A key step in 
applying Theorem 1 to check stability is to ensure that F r {co) and F t {co) have only real 

roots. Such a property can be ensured by using the following result: 

Theorem 2: Let M and L denote the highest powers of s and e s , respectively, 
in F * (s) . Let 77 be an appropriate constant such that the coefficients of the 

terms of highest degree in F r (co) and F^a) do not vanish at co = rj. Then for 

the equations F r (o>) = 0 or F^co) = 0 to have only real roots, it is necessary and 

sufficient that in the intervals 

-2l7t + r]<cQ<2l7t + Ti, / = 1,2,3>K 

F r (co) or F ( (co) have exactly ML + M real roots starting with a sufficiently large /. 

[0070] It is now possible to determine analytically the region in the k ( parameter 
space for which the closed-loop TCP/AQM plant is stable. 

[0071] First analyze the system given by (17) without time delay, i.e., d Q = 0. In 
this case, the closed-loop characteristic equation of the system is given by 
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F(s) = v: 2 +s-kk r (20) 

[0072] For this second-order polynomial, it can be determined necessary and 
sufficient conditions that the controller and the plant parameters have to satisfy to 
guarantee the stability of the delay-free, closed-loop system. Solving the characteristic 
equation (20) for the roots, it follows that 

Su=1 liS. (21) 

[0073] Given t hat r > 0 a nd k > 0 a re always t rue for t he TCP/AQM p lant, t he 
closed-loop delay-free system is stable for the following condition 

k;<Q. (22) 

[0074] Next, let us consider the case where the time delay of the plant model is 
greater than zero, i.e., d 0 >0. The closed-loop characteristic equation of the system is 
then 

F(s) = -kkfi*** + (1 + rs)s . (23) 

[0075] It is necessary to determine the set of all stabilizing gains for the system. 

[0076] In order to study the stability of the closed-loop system, it is necessary to 
determine if all the roots of (23) lie in the open left half plane. The presence of the 
exponential term e' d ° 5 results in the number of roots of F(s) being infinite and this 
makes the stability check very difficult. However, Theorem 1 can be invoked to 
determine the set of all stabilizing gains . This procedure is explained as follows. 

[0077] First consider the quasipolynomial F*(s) defined by 

F*(s) = e d * s F(s) = -kk ( + (1 + Ts)se d ° s . (24) 

[0078] Substituting s^jco, and using the relationship e dJ ° = cos(</ 0 <y) + js\n{d 0 co) , 
the following expression results 
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F*(» = F r (fi>)+y^) 
where 

F r (a>) = -kk> - co sm(d 0 co) - rco 2 cos(d Q o)) 
F^co) = a)[cos(d 0 co) - tcd sin(d 0 6))] 

[0079] For convenience of analysis, make the following change of variables, 
z = d 0 o) . Thus, it is possible to rewrite the real and imaginary parts of F*(ja)) as 



1 



F r (z) = -kk i zsin(z) -z cos(z) 

d n dl 



d 0 v ' dl 



1 * .2, 



F;(z) = — zcos(z) — ~z sin(z) 



(25) 



(26) 



[0080] Theorem 1 requires a check of two conditions to ensure the stability of the 
quasipolynomial F*0). 

[0081] Check of Condition 2 of Theorem 1 : First check if 

E(a> 0 ) = ^ 0 )F r K) - ^K)^ 0 ) > 0 
for some to 0 in (-00,00). 

[0082] Taking <y o = z o = 0, for instance, gives F r (z 0 ) = -kk i and i^(z 0 ) = 0. Also 
obtained is 

i*(*o) = 0 



and 



d 0 -rz 

< d o ) 



cos(z) 



2r + */ n 



zsin(z), 



[0083] 



From these it may be seen that 



-20- 



[0084] Given that for the TCP/AQM plant the conditions r > 0 and k > 0 hold true, 
it is required that ^(20) = -^, >0 ( /e -. <0) for E(z 0 )>0. This result is consistent 
with that given in (22). 

[0085] Check of Condition 1 of Theorem 1 : Now check the interlacing of the 
roots of F r (z) and i^.(z). The roots of the imaginary part, i.e., F i (z) = 0 can be 

determined from (26). This yields 



4, 



X 

cos(z) zsin(z) 

d o 



= 0. 



[0086] It may be seen from this equation that z = 0 is a root of F.(z) , or 

cot(z) = — z (27) 

[0087] One root of the imaginary part F s (z) is z 0 =0 but the other roots are 
difficult to find and require an analytical solution of (27). However, a plot of the two terms 

in (27), (/.e., cot(z) and — z) can be used to understand the nature of the real solution. 

It is possible to plot these two terms for arbitrarily selected values of k , r and d Q . The 
general nature of the plots, however, does not change with the use of other values. 
Arranged in increasing order of magnitude, denote the positive real roots of (27) by z } , 

j = 1,2,3,K From FIG. 3, it may be clearly seen that the non-negative real roots of the 
imaginary part F { {z) satisfy 

z, e (0, |), z 2 e (a, y), z 3 e (2^, y), z 4 e Qtt, ^\ K (28) 
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[0088] Theorem 2 may now be applied to check if F,(z) has only real roots. 
Substitute s l = d 0 s in the expression for F*(s) (in (24)). It may now be seen that the 

new quasipolynomial in yields M = 2 and 1 = 1. Choose tj = — to satisfy the 

4 

requirement that F ( (z) does not vanish at a = rj , i.e., sm(rj) * 0 . From FIG. 3, it may be 

seen that in the interval [0, 2n -— ] = [0, — ] , F ( (z) = 0 has 3 real roots including a root at 

4 4 

the origin z 0 = 0. It is further apparent that F t (z) is an odd function. It therefore follows 

Ik l7t 

that in the interval [ , — ] , F,(z) will have 5 real roots. It is also possible to observe 

4 4 

that F;(z) has one real root in ( — , — ]. Thus, F(z) has 4I + M = 6 real roots in the 

4 4 

interval [-2n + —, 2n+—]. Furthermore, F,(z) has 2 real roots in each of the intervals 
4 4 

[2ln +-,2(1 + \)n + -] and [-2(1 + \)n + -,-2ln + -] for / = 1,2,3,K Hence it follows that 
4 4 4 4 

^.(z) has exactly 4/Z + M real roots in the interval [-2ln + 2ln + -] for / = 1,2,3,K , 

4 4 

which by Theorem 2 implies that F.(z) has only real roots. 

[0089] F r (z) can now be evaluated at the roots of the imaginary part F t (z) . For 
z 0 = 0 , it was obtained previously F r (z 0 ) = -kk t . Now using (25), the following obtains for 

z jt y = l,2,3,K , 

1 r 
F r (z .) = - — z . sin(z y ) - — z 2 cos(z ,) . 

z cotfz^ 

[0090] Using — = — — from (27) in the above equation, it obtains that 

d 0 t 
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F r (zj) = -kk- 



sin(z.)cot(z) x 2 . . 
— ---rzjcosizj) 



F r (z j ) = -kk i - — 



z 2 +^- 

J T 2 



(29) 



cos(z y ) 



[0091] 



where 



Therefore 



F r (z y ) = -*[*,- + a(z,.)], 



(30) 



z 2 +^- 

J T 2 



cos(z y ) . 



(31) 



[0092] From the previous results, F r (z 0 ) = > 0 (i.e., k t < 0 ), then interlacing of 
the roots of F r (z) and F f (z) is equivalent to F r (z,) < 0 , F r (z 2 ) > 0 , F r (z 3 ) < 0 , and so on. , 
Using this fact, the result F r (z 0 ) = -kk. > 0 and equation (30), it may be seen that 



F r (z 0 )>0 => *,<0 

F r (z,)<0 => k i >-a(z l ) = a l 

F r (z 2 ) > 0 ^ kg < -a(z 2 ) = a 2 

F r (z 3 )<0 => k i >-a(z i ) = a 3 
M 



(32) 



[0093] The following observations can now be made (noting that for the 
TCP/AQM plant the x > 0 , d 0 > 0 and k > 0 hold true): 

Observation 1: From (28), it is seen that z y for odd values ofy (1, 3, 5, ...) 
are in the first quadrant. Thus, for odd values of /, cos(z y ) > 0 and from (31), 
it may be concluded that a(zj) > 0 for odd values of the parameter ;'. 

Observation 2: Similarly, it is seen from (28) that z y for even values of /, 
(2, 4, 6, ...) are in the third quadrant. Thus, for even values of /, cos(z y ) < 0 
and from (31 ), it follows that a(z y ) < 0 for even values of the parameter /. 
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[0094] All the set of inequalities obtained above can now be rewritten as 

max {<?,■}<&, < min {a,}. (33) 

y=l,3.SJC J y=0,2,4,6.K 7 

[0095] From (27) and the relationship 1 + tan 2 (z y ) = sec 2 (z y ) , it follows that 



cos(z .) = ± = ±-j=^ 

' l(l + tan 2 (z.) 



(34) 



[0096] From the foregoing analysis z., y = l,2,3,K are solutions of (27). Now 
using the above expression, it is possible to rewrite a(z y ) defined in (31 ) as 

°^- + -k*'\t4 (35) 

where according to Observations 1 and 2 above, the plus sign (+) is used for odd values 
of j\ and the minus sign (-) is used for even values of j. 

[0097] With (35), it is possible to make the following observation: 

Observation 3: Since z, <z 2 <z 3 <z 4 <K , from (35) (and Observations 1 

and 2), it is seen that a(z y ) is a monotonically increasing function for odd 

values of j (1, 3, 5, ...), a(z 1 )<a(z 3 )<a(z 5 )<K , and a(z y ) is a 

monotonically decreasing function for even values of j (0, 2, 4, 6, ...)» 
a(z 0 ) > a(z 2 ) > a(z 4 ) > a(z 6 ) >K . 

It is also clear that a(z 0 ) = 0 , z 0 = 0 . 
[0098] From this observation, the bounds for jfc, in (33) can now be expressed as 

"-fl(z 1 )<* l <0, z,€(0,|). (36) 
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[0100] Therefore, for the interlacing property to hold it must be that 

+ <k, < 0, z, e (0, |). (37) 

[0101] Note that for the values of A:, in this range, the interlacing property and the 

fact that the root of F t {z) are all real can be used in Theorem 2 to guarantee that F r (z) 

also has only real roots. At this point, it is seen that all the conditions of Theorem 1 are 
satisfied. 

[0102] It is possible to summarize the algorithm for determining the integral 
controller parameters as shown in FIG. 4. The process initiates, at step 400, by 
obtaining for a network a set of parameters k, d 0 , and t, where it is a steady-state gain of 
said network, d 0 is a time delay of said network, and t is a time constant of said network. 

Next, at step 410, determining the rootz/ in the interval z, e (0, — ) of 

cot(z) = ^-z , 
«o 

Next at step 420, computing the range of stable gains for it, for this particular network as 

-4 z, r*? <t,<0 • 

[0103] As an example illustrating the use of the algorithm, consider the sample 
problem of characterizing the stabilizing integral controller gains for a TCP/AQM system 
with nominal operating parameters of T = pC = 45 Mbps, iv = 800, and d 0 = 55msec. We 

also select a small nominal packet size of 64 bytes which we use only for the 
computation of t he s tability region. A s mailer p acket s ize i s s elected for t his p urpose 
because of the fluid-flow model used in the TCP/AQM control problem. For this system 
A = 1.0515at10 6 and t = 0.1 652. Note that in deriving the control parameters, T is the 
control target and the TCP parameter 0 = 2/3 . Following the procedure described in 
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FIG. 4, compute z, e (0,—) satisfying (27) to obtain z, =0.54583. From this, it is possible 

2 

to obtain the range of stable k g gains as - 1.814 x io -5 <*, <0. 

[0104] As a simple assessment of the region of gains obtained, the interlacing 
property and the system step response of a gain chosen from within the region may be 
examined. Set the controller gain k i (inside the stabilizing range) as -l.SxlO" 6 . 

[0105] With this value of k n the characteristic quasipolynomial of the system is 
given by 

F*(s) = \ .5773 + (1 + 0.1 652s)se dQS . 
[0106] Substituting s = jo , it follows that 

F * = [l .5773 - co sin(rf 0 <y) - 0.1652<y 2 cos(rf 0 <y)] 
+ /j^cos(rf 0 ^)-0.1652# 2 sin(rf 0 <y)] 

[0107] FIG. 5 shows the plot of the real and imaginary parts of F*(jo)) . It can be 
observed that the real and imaginary parts interlace. 

[0108] Further, it is possible to plot the resultant step response of the closed-loop 
TCP/AQM system as shown in FIG. 6. It can be seen from this figure that the closed- 
loop system is stable and the system output R(t) tracks the step input signal T(t) . 

[0109] Choosing which gain is most appropriate for a given system is a decision 
of the system engineer. Having available a range over which all gains are stable 
provides the advantage to the system engineer of being able to emphasize whichever 
aspect of controller performance is most desired, while at the same time ensuring that 
the system performance remains within bounds. 

[0110] Once the stabilizing integral controller gains have been determined (based 
upon a TCP/AQM model using small nominal packet size, e.g., 64-byte packets), the 
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integral control algorithm can easily be constructed. The integral controller output is 
given in the s-domain as 

P i (s) = C i (s).E(s) = ^-.E(s) . 

5. 

where E(s) is the Laplace transform of the control error e(t) = T - R(t) . 

[0111] Denoting t m , /m = 1,2,3,K as the sampling instants (/'.e. f the times the 

controller reads the input values), then in the continuous-time domain, the controller 
output is given as 



[0112] It follows that 



Pi (t) = k i je(h)dh 



[0113] If this derivative is approximated by the backward difference, the following 
discrete-time approximation may be obtained 

at 

where At = t n - t m _ t , m = \,2, 3,K is the sampling interval. 

[0114] This leads to the following recursive equation for the l-controller 

P ( (t m ) = P i (t M _ l ) + k l Ate(t m ). 

[0115] A flow chart for the control algorithm and the packet mark/drop routine are 
shown in FIG. 7 and FIG. 8. In the figures, the discrete sampling instants are simply 
represented as n = 0A/,lAf,2Af,3A/,K . Note that no packet marking/dropping takes 

place when R(t) < T L in FIG. 8. The parameter T L rate threshold is simply a threshold 
lower than or equal to the control target T during which no packets are marked/dropped 
even though the mark/drop probability p(t) can be greater than zero. This simple 



-27- 



mechanism helps to further minimize oscillations in link utilization and keep the 
utilization around the control target. Given that the models used in the design of the 
control system are approximations of the real TCP/AQM plant {i.e., neglect TCP slow 
start, timeouts), additional mechanisms may be useful to enhance the performance. 

[0116] It is also important to note that rate measurements are usually noisy and 
as a result will require some low-pass filtering. A simple exponentially weighted moving 
average (EWMA) filter, for example, can be used for this. The EWMA filter can be 
expressed as 

R\t m ) = (l- fi)R' ('„-, ) + m m ) - 0 < fi < 1 . 

[0117] As shown in FIG. 7 the process initiates, at step 700, at discrete time n=0, 
by initializing certain parameters. The timer is set to At time units, and mark/drop 
probability p(0), and Rate signal R(0) are set to initial values. The initial mark/drop 
probability is used in the mark/drop routine until further samples are available. A nominal 
packet size of x is also chosen at step 700. At step 710, the timer is reset to At time units 
to advance to the next discrete time interval. Then at step 720, the data arrival rate R is 
measured. 

[0118] At step 730, there is an optional step of pre-filtering the data arrival rate as 
described previously. 

[0119] At step 740, the assigned capacity is determined. Typically, this is a given 
for a particular network configuration but may vary as circumstances warrant, for 
example, if the network is modified. 

[0120] At step 750, an error signal e(n) is calculated as the difference between 
the assigned capacity and the measured (and possibly filtered) data arrival rate The 
error signal is normalized by the nominal packet size x so that the measurements R and 
T will be consistent with the packet size x used in the computations of the stability gain. 
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[0121] At step 760, a current mark/drop probability p(n) is calculated as per an 
integral controller using the stable gain k { established at step 700. 

[0122] The mark/drop probability calculated at step 760 may be used as the 
mark/drop probability until the next measurement time as tracked by the timer, at which 
point a new mark/drop probability will be calculated. In addition, the filtered arrival rate 
R'(n), if filtering is used, is stored to be used at the next measurement time. 

[0123] The process may then loop back to step 710 upon timer expiration for 
another iteration of the process. 

[0124] FIG. 8 presents a flowchart of a random number generator mark/drop 
routine. The decision to accept or mark/drop an incoming packet in the routine is based 
upon the outcome of a comparison of a randomly generated number p r e [0,l] and the 
mark/drop probability p(n). The procedure can be described as follows. 

[0125] Upon a packet arrival at the queue, at step 800, a determination is made 
whether the arrival rate is less than or equal to a no-mark/drop rate threshold. If the rate 
is less than or equal to the rate threshold, then the incoming packet is queued at step 
840. If the rate is not less than or equal the rate threshold, then the routine moves to 
step 810 where a random number p r e[0,l] is generated. At step 820, a determination 
of whether the random number p r is less than or equal to the calculated mark/drop 
probability p(n) is made. 

[0126] If the probability p r is less than or equal, then the packet is 
marked/dropped at step 830. If not, the packet is queued at step 840. 

[0127] Note that the rate measurements (R(t) and the target rate T) can be done 
in bytes or bits per second. The error signal, however, has to be normalized by the 
(small) nominal packet size used in the computing of k, . 
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[0128] A network node using the above rate-based AQM mechanism is able to 
provide high network utilization, absorb transient traffic bursts, while maintaining small 
queues. Following is a description of an enhanced algorithm which retains all the 
features of the basic algorithm in addition to the ability to drop low-precedence packets 
in preference to higher precedence packets. 

[0129] The enhanced algorithm consists of multiple instances of the basic 
algorithm. Each instance is associated with a precedence grade. A precedence grade 
is a traffic type having an associated priority. An example precedence grading scheme 
"is that of the green-yellow-red color schema. Each instance (one for each priority or 
color) will be controlling traffic in a single queue but with the instances having different 
rate congestion indicators and thresholds. As an example of the enhanced algorithm 
applied to a case of three precedence grades, the following parameters are defined: 

• Three rate m easurements (used as congestion indicators), one for each color, 
c e {9 = green, y = yellow, r = red} , with green packets having higher precedence 
than red or yellow packets, and yellow packets having higher precedence than 
red packets: 

o R g « rate of only green packets to the aggregate queue 

o R y * rate of both yellow and green packets to the aggregate queue 

o R r * rate of all packets (of all colors) to the aggregate queue 

Especially note that the rate for a particular color or precedence grade is not the 
specific r ate o f t he p ackets h aving t hat color, b ut f or t he p urposes h ere i s t he 
combined rate of the rate of the packets of that color summed with the rates of 
the packets of higher precedence grade. Thus, R g being the rate of the highest 
precedence grade, is solely the rate of the green packets whereas R y is the sum 
of the rates of the yellow packets plus rate of the green packets, the green 
packets being of higher precedence grade than the yellow. The same relation 
holds for the case of other than three precedence grades. 



-30- 



• Long-term averages of these rates are maintained as R g , R y9 and R r% 
respectively. 

• Three rate thresholds capacities, one for each color, T c , ce {g,y,r} , as illustrated 
in FIG. 9. 

• Three no-drop thresholds, one for each color, T L c , c e {g,y,r} 

• Three drop probability computations, one for each color, p i c , c e {g,y,r} 

PiM) = PiM - l ) + k A te M) . c e {g>ys} 

[0130] The enhanced algorithm operates such that under sustained congestion 
(where traffic of all color are sent to the queue), the drop probabilities p ig , p iy and p ir 

will increase relative to each other as shown in FIG. 10. At any given instant during the 
congestion, the drop probabilities will generally be p ig < p i y < p i r . 

[0131] The drop probabilities are computed as shown FIG. 11. The process 
initiates, at step 300, at discrete time n=0, by initializing certain parameters. The timer is 
set to At time units, and mark/drop probability p(0), and Rate signal R c (0) are set to initial 
values. Note that there is a Rate signal R c for each of the colors (precedence grades). 
An integral controller gain /c, is specified. The initial mark/drop probability is used in the 
mark/drop routine until further samples are available. A nominal packet size of x is also 
chosen at step 300. 

[0132] At step 310, the timer is reset to At time units to advance to the next 
discrete time interval. Then at step 320, the data arrival rate R c is measured appropriate 
to a particular color. 

[0133] At step 330, there is an optional step of pre-filtering the data arrival rate as 
described previously. 
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[0134] At step 340, the assigned capacity for the color under consideration is 
determined. Typically this is a given for a particular network configuration but may vary 
as circumstances warrant, for example, if the network is modified. 

[0135] At step 350, an error signal e c (n) is calculated as the difference between 
the assigned capacity and the measured (and possibly filtered) data arrival rate for that 
color. The error signal is normalized by the nominal packet size x so that T c and the 
measurement R c will be consistent with the packet size x used in the computations of 
the stability gain. 

[0136] At step 360, a current mark/drop probability Pi (C (n) is calculated as per an 
integral controller using the stable gain ft* established at step 300. 

[0137] The mark/drop probability calculated at step 360 may be used as the 
mark/drop probability until the next measurement time as tracked by the timer, at which 
point a new mark/drop probability will be calculated. If filtering is used, the filtered arrival 

rate R c (n) is stored to be used at the next measurement time. 

[0138] The process may then loop back to step 310 upon timer expiration for 
another iteration of the process. 

[0139] FIG. 12 illustrates the packet drop routine for packets arriving at the 
queue. Given that under sustained traffic with all colors, it can be seen that R g <R y < R r 

and given that T g >T y >T r , generally, red packets are dropped the most and green 

packets are dropped the least. Green packets are dropped only under severe 
congestion. The decision to accept or mark/drop an incoming packet in the routine is 
based upon the outcome of a comparison of a randomly generated number p r e [0,l] 
and the mark/drop probability p(n). The procedure can be described as follows. 

[0140] Upon a packet arrival at the queue, at step 500, a determination is made 
whether the arrival rate for that color (precedence grade) R c {n) is less than or equal to a 
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corresponding no-mark/drop rate threshold ^ c f° r that color. lf the rate is ,ess than or 
equal to the rate threshold, then the incoming packet is queued at step 540. 

[0141] If the rate is not less than or equal the rate threshold, then the routine 
moves to step 510 where a random number p r e[o,l] is generated. 

[0142] At step 520 a determination of whether the random number p r is less 
than or equal to the calculated mark/drop probability p iyC (n) is made. 

[0143] If the probability p r is less than or equal p ilfi (n) 9 then the packet is 
marked/dropped at step 530. If not, the packet is queued at step 540. 

[0144] Note that the rate measurements (R c (n) and the target rate T c ) can be 
done in bytes or bits per second. The error signal, however, has to be normalized by 
the (small) nominal packet size used in the computing of k im 

[0145] The example illustrated in FIGURES 9 to 13 illustrates the case for three 
precedence grades. An elaboration to a different quantity of grades is contemplated, 
such as a pair of grades where a simpler differentiation of service is desired (e.g., high 
grade versus low grade) or, alternatively, a quantity higher than three where additional 
differentiation would provide for finer gradations. 

[0146] In an alternative contemplated embodiment, a simplified set of thresholds 
is used. Specifically, applied to a case of three precedence grades, the following 
parameters are defined: 

• Three rate measurements (used as congestion indicators), one for each color, 
ce {g = green, y = yellow, r = red} 

o R g * rate of only green packets to the aggregate queue 

o R y * rate of both yellow and green packets to the aggregate queue 
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o R r * rate of all packets (of all colors) to the aggregate queue 

Long-term averages of these queue sizes are maintained as R g , R y , and R r , 
respectively. 

• One rate threshold capacity 410 for all colors, T = T c , c e {£,>>,r} , as illustrated in 
FIG. 13 

• One no-drop threshold for all colors, T L = T L c , c e {g,y,r} 

• Three drop probability computations, one for each color, p ic , c e {g,y 9 r} 
P itC (n) = PiA n - l ) + k A te c( n )> ce{g,y 9 r} 

[0147] As with the previous embodiment, the drop probabilities will increase 
relative to each other as shown in FIG. 10. At any given instant during the congestion, 
the drop probabilities will be p i g < p i y < p i r . Given that under sustained traffic with all 

colors, we have R g <R y < R r and given that T g = T y = T r , generally, red packets are 

dropped the most and green packets are dropped the least. Green packets are dropped 
only under severe congestion. The use of a single threshold may serve to simplify the 
implementation over the multiple threshold variation. 

[0148] While the invention has been described in conjunction with specific 
embodiments thereof, it is evident that many alternatives, modifications, and variations 
will be apparent to those skilled in the art in light of the foregoing description. 
Accordingly, it is intended to embrace all such alternatives, modifications, and variations 
as fall within the spirit and broad scope of the appended claims. 



